ON POLARIZATIONS IN INVARIANT THEORY 
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Abstract. Given a reductive algebraic group G and a finite dimensional algebraic G- 
module V, we study how close is the algebra of G-invariant polynomials on V® n to the 
subalgebra generated by polarizations of G-invariant polynomials on V . We address this 
problem in a more general setting of G-actions on arbitrary affine varieties. 



1. Introduction 

1.1. Let G be a reductive algebraic group over an algebraically closed field k of charac- 
teristic 0, and let V be a finite dimensional algebraic G-module. Given a positive integer 
n, consider the G-module V® n := V . . . V (n summands). Finding generators of the 
invariant algebra k[V® n ] G of V® n is the classical problem of invariant theory. The clas- 
sical method of constructing elements of /c[y® n ] G is taking the polarizations of invariants 
/ € fc[V] G , i.e., the polynomial functions ... j n on V® n given by the formal expansions 

f{xiVl + ...+ X n V n ) = Ei 1) ...,i n6 Z + X l ■ ■ ■ X n fh,-M( V li ■■■> V n)> i 1 ) 

where (v±, . . . ,v n ) is generic element of V® n and variables. Let poL^V] 

be the subalgebra of fc[y® n ] G generated by the polarizations of all the /'s. 
There are G-modules enjoying the property 

Vo\ n k[V] G = k[V® n f. (2) 

For instance, (2) holds, by Study's theorem, [St], for the standard action of G = O m on 
V = k m . By Weyl's theorem, [We], (2) holds for G = S m acting on V = k m by permuting 
the coordinates. In [Hun], (2) is established for the natural action of the Weyl group G of 
type B m on V = k m and for the standard action of the dihedral group G on V = k 2 . 

However, in general, pol n A:[y] G ' and fc[F® n ] G ' do not coincide. For instance, for the 
natural action of G = SL n on V = k n clearly pol n A;[y] G = k (since /c[y] G = k) but 
&[y® n ] G zfz k. It is less easy to find examples where (2) fails for finite G, but such examples 
exist as well: in [Wa] it was observed that (2) does not hold for the natural action of the 
Weyl group G of type D m on V = k m (m ^ 4) for n 2. 

In this paper we analyze the relationship between /c[l/® n ] G and pol n k[V] G . We prove that 
if G is finite, then /c[y® n ] G is the integral closure of pol n A;[y] G in its field of fractions, and 
the natural morphism of affine varieties determined by these algebras is bijective. Actually, 
instead of linear actions we consider the more general setting of actions on arbitrary affine 
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varieties for which we define a generalization of polarizations. In this setting, we prove 
that if G is finite, then the invariant algebra is integral over the subalgebra generated 
by generalized polarizations, and the natural dominant morphism between affine varieties 
determined by these algebras is injective (in the graded case, bijective). 

For connected G, one cannot expect such results, as the example of SL n acting on k n 
shows. This naturally leads to distinguishing the n's for which fc[V® n ] G is integral over 
pol n fc[V] G and defining the polarization index ofV, 

pol ind(y), 

as the supremum taken over all such n's. We prove that /c[V® m ] G is integral over pol m /c[V] G 
for every m ^ pol ind (F), and show that calculating pol ind (y) is closely related to the 
old problem of describing linear subspaces lying in the Hilbert nullcone of V (see [Hil], 
[MFK], [PV], [G], [CRT], [MOR], [MFK], and the references therein), namely, to analyzing 
a certain geometric property of such subspaces. 

Using this reduction, we calculate the polarization index of some G-modules V. Namely, 
we prove that if G is a finite group or a linear algebraic torus, then pol ind (V) = oo. For 
G = SL2, we describe all linear subspaces of V lying in the Hilbert nullcone of V and 
prove that pol ind (V) = 00 if V does not contain a simple 2-dimensional submodule, and 
pol ind (V) = 1 otherwise. Finally, we calculate the polarization index of every semisimple 
Lie algebra q: we prove that pol ind (0) = 1 if q is not isomorphic to 3(2 © ... © 0(2, and 
pol ind (g) = 00 otherwise. As an application to the above mentioned old topic of linear 
subspaces lying in the Hilbert nullcone, we prove that a semisimple Lie algebra q contains a 
2-dimensional nilpotent nontriangularizable linear subspace if and only if q is not isomorphic 

to Sl 2 © • • • ©5l 2 . 

1.2. Notation. k[X] is the algebra of regular functions of an algebraic variety X. If X is 
irreducible, k(X) is the field of rational function of X. 

If a group S acts on a set Z, we put Z s := {z G Z \ s ■ z = z for all s G S}. 

Below every action of an algebraic group is algebraic (morphic). 

G° is the identity component of an algebraic group G. 

If X is an affine variety endowed with an action of a reductive algebraic group G, then 
tt xg : X — ► X//G is the categorical quotient, i.e., X//G is an affine algebraic variety and 
ir XG & dominant (actually, surjective) morphism such that tt x G (k[X // G]) = k[X] G . 

Given a linear algebraic torus T, its character group Hom(T, G m ) is written additively. 
The value of A G Hom(T, G m ) at t G T is denoted by t x . For an algebraic T-module V 
(not necessarily finite dimensional), V\ is the A-isotypic component of V, 

V\ := {v £ V \ t ■ v = t x v for every t G T}. 

By (v 1, . . . , v n ) we denote the linear span of vectors v±, . . . ,v n of a vector space over k. 
We set N := {1, 2, . . .} and Z+ := {0, 1, 2, . . .}. 

2. Generalized polarizations 

2.1. Let a reductive algebraic group G act on the irreducible affine algebraic varieties X 
and Y. Let Z be an irreducible affine algebraic variety endowed with an action of a linear 
algebraic torus T. The set A := {A G Hom(T, G m ) \ k[Z}\ ^ 0} is then a submonoid of 
Hom(T, G m ) and the isotypic components yield a A-grading of k\Z\. 

k[Z] = AeA k[Z] x , k[Z]^k[Z] u C k[Z]^ +u for all G A. (3) 
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The groups G and T act on Y x Z through the first and second factors, respectively. 
From k[Y x Z] = k[Y] <g> k[Z] and (3) we obtain 

k[Y x Z] = AeA k[Y] k[Z] x and k[Y x Z] G = ® XeA k[Y] G ® k[Z] x . (4) 

We identify k{Y] and k[Z] respectively with the subalgebras A;[y]<g)l and l(8)fc[Z] of k[Y xZ]. 

Assume now that there is an open T-orbit O in Z. This condition is equivalent to either 
of the following properties (ol), (o2), see [PV, Theorem 3.3]: 

(ol) k(Z) T = k; 
(o2) dimfefZ^ = 1 for every A € A. (5) 

For every A € A, fix a nonzero element b\ G k\Z\\. Multiplying every b\ by an appropriate 
scalar we may assume that 

bfj,b u = b^ +u for all fi, u G A. (6) 

Indeed, fix a point xq G O. The definition of k[Z]\ implies that b\(xo) / 0, so replacing b\ 
by b\/b\(xo) we may assume that b\(xo) = 1. Then (6) follows from (5), (3). 

From (4) and (5) it follows that every h G k[Y x Z] admits a unique decomposition 

h = ZxeAPxh, Px^k[Y] (7) 

(in (7) all but finitely many p\s are equal to zero), and h lies in k[Y x Z] G if and only if 
Px G fe[y] for all A. From (6) we obtain 

(E,eAPM &veAP'X)= ZxeA&»+.=xP>"»)bx, P>'l € k[Y}. (8) 
Consider now a G-equivariant morphism 

if : Y x Z — ► X. (9) 

Definition 2.2. Let / G k[X] G . The invariants px G k[Y] G defined by (7) for h = f*(f) are 
called the if -polarizations of f. The subalgebra of fc[y] G generated by all the (^-polarizations 
of the elements of /c[A] G is denoted by pol^/c[A] and called the if -polarization algebra ofY. 

Remark 2.3. More generally, if $ is a collection of G-equivariant morphisms (9) (where Z 
and X depend on ip), then one can define the & -polarization algebra ofY as the subalgebra 
of fc[y] G generated by all the ^-polarization algebras of Y for tp G <£. 

Since changing the b\s clearly replaces the (/9-polarizations of / G fc[A] G by their scalar 
multiples, the algebra pol^/c[A] G does not depend on the choice of the &a's. 

Example 2.4. If Z is a single point, (9) is a morphism ip : Y — > X, and pol^/c[A] G = 
p*(k[X] G ). 

Example 2.5. (Classical setting) Let V be a finite dimensional algebraic G-module and 
let n G N. Take X = V and Y = V® n with the diagonal G-action. Let Z be A n endowed 
with the natural action of the diagonal torus T of GL n , 

diag(ti, . . . ,t n ) ■ (ai,..., a n ) = {hai, . . . , t n a n ). 

Identifying . . . , i n ) G Z n with the character T — > G m , diag(ti, . . . , i n ) i— > t^ 1 • • • i^, 
we identify Z™ with Hom(T, G m ). Then A = Z" . If zi, . . . , z n are the standard coordinate 
functions on Z, then for every A = . . . , i n ) G A, the isotypic component A;[Z]a is spanned 
by b\ := zl 1 ■ ■ ■ z l ™ . So, condition (5) holds. Clearly, (6) holds as well. 
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Recall that the classical n-polarizations of a polynomial / G k[V] are the polynomials 

fh,...,i n G fc[F ffiri ], where (zi, . . . ,i n ) G A, such that 

/(£"=! W) = £i lv ..,i n eA «i • • • < n /n,..,in(^i, • • • ,«n) for all Vj G V, ay G fc. (10) 

Since o^ 1 • • • a^ 1 is the value of z 1 ^ ■ ■ ■ z % ™ at (a±, . . . , a n ) G Z, it readily follows from (10) 
and Definition 2.2 that the classical n-polarizations of / are the (^-polarizations of / for 

if := T n : Y x Z -> X, ((vi, . . . , v n ), (ai, . . . , a n )) i-» aivi + . . . + a n v n . (11) 

In this setting, we denote the 93-polarization algebra of Y by pol ra fc[F] G . □ 

Example 2.6. If G = O m and V = k m with the natural G-action, then pol n /c[y] G = 
k[V® n ] G by Study's theorem, [St]. 

If G = Sp m , m even, and V = k m with the natural G-action, then pol TnXTn k[V® 2 ] G = 
k[V® n } G (see (11)) by [We]. 

If G = SL m and V = k m with the natural G-action, then pol TnX . .. XTn k[V em ] G = 
k[V® n f, see [We]. □ 

From (8) we deduce that the algebra poLpfcpT] is generated by all (^-polarizations of the 
/'s for / running through the generators of fc[X] G . Since by Hilbert's theorem, the algebra 
A;[X] G is finitely generated, this means that the algebra pol^/cfX]*^ is finitely generated as 
well. Hence there is an affine algebraic variety that we denote by Y//ip, and a dominant 
morphism tt v : Y -»■ Y//<p such that n*(k[Y//<p\) = poL p fc[X] G '. Since pol^X} C k[Y] G , 
the definition of categorical quotient for the G-action on Y implies that there is a dominant 
morphism v : Y//G — > Y//ip such that the following diagram is commutative: 

Y Y//G 

^\ ' 

The set of all morphisms from Z to X is endowed with the G-action defined by the 
formula (g ■ ip)(z) := g ■ (ip(z)) for tp : Z — ► X, 51 G G, z G Z. Using (9), we can consider Y 
as a G-stable algebraic family of such morphisms. Namely, with every y G Y we associate 
the morphism 

tfy : Z — ► X, z ^ ip(y, z). (13) 
Then for every z G Z and g £ G we have (Pg-yi z ) = ¥>(s -y,z) = (f(g ■ (y, z)) = g(<p(y, z)) = 
(9 ■ <Py){z), so </><?-y = 9-<Py 

Lemma 2.7. For ewery 2/1,2/2 G Y", i/ie following properties are equivalent: 

(i) ^(Ms/i)) =^ 1 (TT ip (y 2 )); 

(ii) Ta-,g V 3 ?/! = k x ,g ^2/2 • 

Remark 2.8. Property (i) means that points 2/1,2/2 G F are not separated by the </?- 
polarization algebra poLpfc[X] G . 

Proof. By virtue of (13), property (ii) is equivalent to the property 

^gO^S/i^)) = 7r XG (99(2/2,2;)) for all z G Z. (14) 
Since the variety X is affine, for a fixed 2 G Z, equality in (14) holds if and only if 

s Kc(v(yii 2 ))) = s Kc(¥'(!/2. j; ))) for every s € fcpC/G]. (15) 
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Since ir x G (k[X//Gj) = k[X] G , in turn, (15) is equivalent to the property 

f{<p(yi,z)) = f(<p(y2,z)) for every / G k[X] G . (16) 

Setting h = <p*(f) for / in (16), we thus obtain h(y±, z) = h(y2, z) for all zfiZ, i.e., using the 
notation of (7), ^2\ e ^P\(yi)b\ = SagA^A^)^- Since {6a} are linearly independent, this 
shows that the equality in (16) is equivalent to the collection of equalities p\(yi) = P\(y2), 
A G A. Definition 2.2 and Remark 2.8 now imply the claim. □ 

Lemma 2.9. If G is a finite group, then for every two morphisms ipi : Z — > X, i = 1, 2, 
the following properties are equivalent: 

(i) 7r X G o-0i = 7r XiG ,oV> 2 ; 

(ii) i/jere is g £ G such that ip2 = 9 • ipi- 

Proof. (ii)=>(i) is clear (and holds for every reductive G, not necessarily finite). Assume 
now that (i) holds. Consider in Z x X the closed subset 

* := {(z,x) G Z x X | 7r XiG (Vi(z)) = n XjG (x)}. (17) 

Since G is finite, every fiber of G-orbit, see, e.g., [PV, Theorem 4.10]. Hence for 

(z,x) G Z x A, the condition 7r x g (V>i(.z)) = 7r XG (x) in (17) is equivalent to the existence 
of g G G such that x = g ■ (ipi(z)) = (g • ipi)(z). In turn, the last equality means that the 
point (z, x) lies in the graph of g ■ tpi, 

Tg.^ := {(z, (g ■ ^i)(z)) G Z x A | z G Z}. (18) 

On the other hand, (17), (18) clearly imply that Tg.^ C ^ for every 5. Thus, 

* = U 96G r^ 1 . (19) 

But every T g .^ )1 is a closed subset of Z x X isomorphic to Z. So, by (19), \l/ is a union 
of finitely many closed irreducible subsets of the same dimension. Hence these subsets are 
precisely the irreducible components of VP . 
On the other hand, it follows from (i) that 

* := {(z,x) G Z x X I n x , G (Mz)) = «x,g( x )}- ( 20 ) 
Using the above argument, we then deduce from (20) that the graph of 1^2, 

r^ 2 :={{z^ 2 {z))eZxX\zeZ}, 

is an irreducible component of * as well. Therefore there is g G G such that T^ 2 = Tg.^ . 
Hence g ■ tpi = i/j 2 , i-e., (ii) holds. □ 

Theorem 2.10. Maintain the notation of this section. If G is a finite group, then 

(i) the morphism v : Y//G — > Y//(p in (12) is injective; 

(ii) k(Y) G is the field of fractions of the (p -polarization algebra Y>o\^k[X] G . 

Proof. Since G is finite, fibers of tt x G are precisely G-orbits. On the other hand, by Lem- 
mas 2.7 and 2.9, every fiber of tt^ is a G-orbit as well. This and the commutative diagram 
(12) yield (i). Since v is dominant and char k = 0, from (i) it follows that v is a birational 
isomorphism. Since Y//ip is affine, k(Y//(p) is the field of fractions of k[Y//ip\. This and (12) 
now imply (ii). □ 
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2.11. Under a supplementary assumption there is a geometric criterion of finiteness of v. 
It is based on a general statement essentially due to Hilbert. Namely, consider an action 
of a reductive algebraic group G on an irreducible affine algebraic variety M. Assume that 
the corresponding G-action on k[M] preserves a Z + -grading k[M] = ® n & + k[M] n such that 
fc[M]o = k and dim k[M] n < oo for every n. Let A be a homogeneous subalgebra of k[M] G . 

Lemma 2.12. The following properties are equivalent: 

(i) {x G M | f{x) = V/ G ® n&i k[M] G } = { x G M \ h(x) = V/i G ©„ eN A»}; 

(ii) k[M] G is integral over A. 

If these properties hold and G is connected, then k[M] G is the integral closure of A in k[M]. 

Proof. For linear actions, (i)=>(ii) is proved by Hilbert in [Hil, §4]. In the general case 
the argument is the same. Implication (ii)=^(i) is clear. The last statement follows from 
the first since it is well known that k[M] G is integrally closed in k[M] for connected G 
(connectedness of G implies that G acts trivially on the set of roots of the equation of 
integral dependence). □ 

Lemma 2.12 implies the following geometric criterion of finiteness of v. Assume that the 
G-actions on X and Y can be extended to the G x G m -actions such that 

ip is G x G m -equivariant, (21) 

k[X] Gm = k[Y] Gm = k. (22) 

From (22) we then deduce that Hom(G m , G m ) can be identified with Z so that the isotypic 
component decompositions of k[X\ and k[Y] become the Z + -gradings of these algebras, 

k[X] = nez+ k[X] n , k[Y] = nez+ k[Y] n . (23) 

Since every isotypic component is a finitely generated module over invariants, see, e.g., [PV, 
Theorem 3.24], from (22) we deduce that these gradings enjoy the properties 

k[X]o = k[Y]o = k and dimfc[A] n < oo, dim £;[>"]„ < oo for every n G Z + . (24) 

It follows from (21) that k[X) G and k[Y] G are graded subalgebras of the graded algebras 
k[X] and k[Y] respectively, and from Definition 2.2 we deduce that poLp/c[A] G is a graded 
subalgebra of the graded algebra /c[Y] G . 

The ideal ©„ e NA;[A]„ in k[X\ (respectively, (B n ef^k[Y] n in k[Y]) is maximal and G m - 
stable, so the point Ox G X (respectively, Oy G Y) where it vanishes, is G m -fixed. As 
invariants separate closed orbits, see, e.g., [PV, Theorem 4.7], (22) implies that X Gm = 
{Ox}, Y Gm = {Oy}. Hence Ox € X G , Oy G Y G . From (21) we deduce that <p(0y xZ) = Ox- 
We put 

%:=4Kg^))' TV-^MOy)), AA XjG := vr x ; G (vr XiG (0x)). (25) 
By virtue of (12), the following inclusion holds: 

A/y G C V Y<G . (26) 
Since N Y q is precisely the set of points of Y whose G-orbit contains Oy in the closure, 

A/- 5iG = SnA/y G (27) 
for every G-stable closed subset S of Y containing Oy. 
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Example 2.13. Maintain the notation of Example 2.5. Then the G m -actions on X = V 
and Y = V® n by scalar multiplications yield the G x G m -extensions of G-actions such 
that (21), (22) hold. Thus, in the classical setting, the assumptions of Subsection 2.2 hold. 
In this case, Ox = 0, Oy = (0, ...,0). The varieties N v@n G and N VG are respectively 
the Hilbert nullcones of G- modules V® n and V, and V v ® n G is the locus of the maximal 
homogeneous ideal of pol n /c[y] G . □ 

Lemma 2.14. Maintain the assumptions of Subsection 2.2. The following properties are 
equivalent: 

(i) v is finite; 

(ii) M Y = Vy iG . 

Proof. This immediately follows from Lemma 2.12. □ 

Theorem 2.15. Maintain the assumptions of Subsection 2.2 and let G be finite. Then 

(i) v is finite and bijective; 

(ii) if Y is normal, v : Y//G — > Y//ip is the normalization ofY//(p, and k[Y] G is the 
integral closure of pol^fcpr]*^ in k(Y) G ; 

(iii) ifY is normal, pol k[X] G = k[Y] G if and only i/pol^/c[X] G is integrally closed. 

Proof. Theorem 2.10(i) implies that N Y G = ^Vg ( = ^y)- Hence v is finite by Lemma 2.14. 
Being finite, v is closed, and since v is also dominant, Theorem 2.10(i) implies that v is 
bijective. This proves (i). If Y is normal, then Y//G is normal as well, see, e.g., [PV, 
Theorem 3.16]. Since by Theorem 2.10(h), v is a birational isomorphism, this, (i), and the 
definitions of Y//G, Y//ip, v prove (ii). Claim (iii) follows from (ii). □ 

Corollary 2.16. In the classical setting (see Example 2.5), let G be finite. Then 

(i) v is finite and bijective; 

(ii) k[V® n ] G is the integral closure of pol n k[V] G in k(V® n ) G ; 

(iii) fc[y® n ] G = poLj/cfVp if and only ifpol n k[V] G is integrally closed. 

Example 2.17. Maintain the notation of Example 2.5 and let V = k m . If G is the 
symmetric group in m letters acting on V by permuting the coordinates, then fc[V® n ] G = 
pol n k[V] G for every n, [We]. This equality also holds for the Weyl group of type B m and 
the dihedral groups, [Hun]. But for the Weyl group of type D m , m ^ 4, and n = 2 it does 
not hold, [Wa]. 

Namely, D m acts on the standard coordinate functions x\, . . . , x rn on V by permutations 
and changes of an even number of signs, and /c[V] Dm = k[a\, . . . , a m ] where 

a s = XXi x i S for 1 ^ s ^ m - 1, a m = x 1 ■ ■ ■ x m , 

see, e.g., [Hum]. Take another copy of V with the standard coordinate functions yi, . . . , y m , 
and naturally identify x±, . . . , x m , yi, ■ ■ ■ ,y m with the functions on V® 2 . Then fc[F® 2 ] Dm is 
generated by pol 2 fc[V] Dm and the polynomials 

P ri ...P rd (a n ), n,...,r d odd, J2i=i n ^ n - d, (28) 

where P r := YliLiVi^^ see [Wa], [Hun]. The group B m is generated by D m and the 
reflection w such that w • x% = x% for i < m and w • x m = —x m . The operators P n 
from (30) commute with the diagonal action of B m on V® 2 , therefore w(P ri . . . P rd (a n )) = 
-P n . . . P rd (a n ). This yields 

(P ri ...P rd (a n )) 2 ek[V® 2 ] B ™. (29) 
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Since k[V® 2 ] Bm = pol 2 k[V} Bm and, clearly, pol 2 k[V] Bm C pol 2 k[V] Dm , we deduce from (29) 
that fc[y® 2 ] Dm is integral over pol 2 /c[V] Dm . This agrees with Theorem 2.15 (that gives more 
delicate information). 



3. Polarization index 

3.1. In this section we take up the classical setting and maintain the notation of Exam- 
ples 2.5, 2.13, and that of (25). If n,m G N and n < m, we naturally identify V® n with 
the subspace {(«i, . . . , v n , 0, . . . , 0) | v\ £ V} of y® m . It is then not difficult to see that 

V v en :G = v® n n Pyem G . (30) 

Lemma 3.2. The following properties of a point v = (vi, . . . , v n ) G V® n are equivalent: 

(i) v G V v9nG ; 

(ii) {v u ... ,v n ) C J\f VG . 

Proof. Let / G ^[Vp be a nonconstant homogeneous function. If v G ^yenQ, * nen 
definition of P v<SnG (see (25) and Example 2.13) yields that, in the notation of (1), we have 
fh,...,i n ( v ) = for all i\, . . . ,i n G Z+. From this and (1) we obtain 

f{a\V\ + . . . + a n v n ) = for all ctj G k. (31) 

So, (vi, . . . ,v n ) lies in the zero set of every /. The definition of M VG (see (25) and Exam- 
ple 2.13) now implies that (v\, . . . , v n ) C M VG . 

Conversely, assume that the last inclusion holds. By the definition of N VG , this implies 
(31). By (1), this in turn yields that /j 1 ,...,i n (w) = for all ii,...,i n G Z+. The definition 
of Py 8 „ q then implies that v G "P^/enG" d 

Definition 3.3. TTie polarization index of a G-module V is 

pol ind (V) := sup n 

with the supremum taken over all n such that in (26) the equality holds, N v<BnG = "Py©„ G - 

From (25), Definition 3.3, and the equality pol^Vp = k[V) G we obtain 

polind(V) ^ 1. (32) 

It is also clear that 

pol ind(V © U) = pol ind(y) if U is a trivial G-module. (33) 
Lemma 3.4. For every n G N, 

= £yen G «/ n ^ pol ind (V), 

£ ^«)»G */ n > p q1 ind (y)- 



V® n ,G 



Proof. By virtue of Definition 3.3, for n > pol ind (V), this follows from (26), and for 
n < pol ind (V), from (27) and (30). □ 

Corollary 3.5. The extension pol n k [V] Ck[V® n ] G is integral if and only i/n^pol ind(V). 
Proof. This follows from Lemmas 3.4 and 2.14. □ 
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Call a character G m — > G m , t i— > i d , positive if d > 0. Every homomorphism 7 : G m — > G 
endows V with the structure of G m -module defined by t ■ v := j(t) ■ v. We denote by V(^y) 
the submodule of this G m -module equal to the sum of all the isotypic components whose 
weight is positive. Clearly, if v G V(j), then the closure of G m -orbit (and, all the more, 
G-orbit) of v contains 0y. Hence ^(7) C M VG . The Hilbert-Mumford theorem, [Hil], 
[MFK] (see, e.g., [PV, 5.3]), claims that 

AV,G = U T n7)- (34) 

Lemma 3.6. The following properties of an integer n G N are equivalent: 

(i) for every linear subspace L such that dimL ^ n and L C Af VG , there is a homo- 
morphism 7 : G m — > G suc/i i/iai L C V(7)> 

(ii) n ^ pol ind (V). 

Proof. Let (i) hold. Take a point u = (v±, . . . , v n ) G T J vBnG . By Lemma 3.2, (v±, . . . , v n ) is 
contained in M VG . By (i), (t>i, . . . , v n ) C ^(7) for some 7. This implies that the closure of 
G m -orbit (and, all the more, G-orbit) of v contains 0y ffin , i.e., v G J\f v(BnG . So, by (26), we 
have ^Py®n G = -^~y(Bn G i whence n ^ pol ind (V) by Definition 3.3. This proves (i)=>(ii). 

Conversely, let (ii) holds. Consider in M VG a linear subspace L of dimension ^ n. Then 
L = (vi,..., v n ) for some G V. By Lemma 3.2, the point v = (vi, . . . , v n ) lies in Vy^Q. 
By (ii), and Lemma 3.4, we have v G Ny 9nG . From (34) we now deduce that v G V® n {^f) 
for some 7. Since V® n {^j) = V{^)® n , this yields Vi G ^(7) for every i, or, equivalently, 
L C V(7). Thus (ii)=>(i) is proved. □ 

Corollary 3.7. (i) Xei U be a submodule ofV. Then 

pol ind ([/) ^ pol ind (V). 

(ii) Let Vi be a finite dimensional algebraic module of a reductive algebraic group Gi, 
i = l,...,m, and let G = G± x . . . x G m , V = V\ . . . V m . JTten 

pol ind(y) = minpolind(Vi). 

i 

Proof. Statement (i) readily follows from Lemma 3.6. Let the assumptions of (ii) hold, and 
let 7Tj : V — > Vi, Pi : G — > Gj be the natural projections. Since the Hilbert nullcone is the set 
of points whose orbits contain zero in the closure, we have M VG = M v G x . . . x jVy G . 
A linear subspace L lying in M G v is contained in V(j) for some 7 if and only if 7Tj(L) C 
Vi(pi o 7) for every i. Using these properties and Lemma 3.6, we deduce (ii). □ 

3.8. We now calculate the polarization index of some G-modules. 

Theorem 3.9. If G is a finite group, then for any G-module V , 

pol ind (V) = 00. 

Proof. This follows from Theorem 2.15, Lemma 2.14, and Definition 3.3. □ 
Theorem 3.10. If G is a linear algebraic torus, then for any G-module V, 

pol ind (V) = 00. 

Proof. It is well known (and immediately follows from (34)) that in this case there are 
homomorphisms 7« : G m — > G, i = 1, . . . , s, such that 

A/V )G = y(7i)u...uF( 7s ). ( 35 ) 
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Since every linear subspace L of V is an irreducible algebraic variety, (35) implies that if 
L C M VG , then L C V(ji) for some i, whence the claim by Lemma 3.6. □ 

Lemma 3.11. Let k[V} G =k. 

(i) pol ind (V) = sup n where the supremum is taken over all n such that k[V S)n ] G = k. 

(ii) IfG° is semisimple, then pol ind (V) is equal to the generic transitivity degree of the 
G-action on V , see [P], i.e., to the maximum n such that there is an open G-orbit 
in V® n . In this case, 

pol ind (V) < dimG/ dim 

Proof. The condition k\V) G = k and Definition 2.2 imply that V v(BnG = V® n for every n. 
On the other hand, k\V® n ] G = k is equivalent to A/" ye „ G = U® n . This gives (i). Being 
semisimple, G° has no nontrivial characters, hence fc[U® n ] G = k is equivalent to the exis- 
tence of an open G-orbit in U® ra ; see [PV, Theorem 3.3 and the Corollary of Theorem 2.3]. 
This proves (ii). □ 

Example 3.12. If G = SL m and V = k m with the natural G-action, then Lemma 3.11 
implies pol ind (V) = m — 1. 

If G = Sp m and V = k m (m even) with the natural G-action, then Lemma 3.11 implies 
pol ind (V) = 1. 

If G = O m and V = k m with the natural G-action, then Example 2.6 and Definition 3.3 
yield pol ind (V) = oo. 

If G = SO m and V = k m with the natural G-action, then the classical description of 
k[V® n ] G , see [We], implies that k[V® n ] G is integral over k[V® n }° m . Hence in this case 
again pol ind (V) = oo, however, in contrast to the case of O m , the algebras /c[V® n ] G and 
pol n /c[l/] G do not coincide if m divides n. □ 

We now calculate the polarization index of any SL2-module. Denote by the SL2- 
module of binary forms in x and y of degree d, see, e.g., [PV, 0.12]. Up to isomorphism, 
Rd is the unique simple SL2-module of dimension d + 1. According to the classical Hilbert 
theorem, [Hil, §5] (see, e.g., [PV, Example 1 in 5.4]), 

A^sl 2 = Uie Rl l [d/2]+1 R d - W 2]-i, (36) 
and for every nonzero I G R\, there is a homomorphism 7 : G m — > SL2 such that 

l [d/2]+1 R d - W 2]-i = Rd(l) (37) 

and vice versa. 

Lemma 3.13. For d ^ 2, the following properties of a linear subspace L of Rd lying in 
SL 2 are equivalent: 

(i) L is maximal (with respect to inclusion) among the linear subspaces lying inN RdSlj2 ; 

(ii) there isleR 1 ,l^0 such that L = l [d/2]+1 R d -[d/2]-i- 

Proof. Using that k[x,y] is a unique factorization domain and every / G R\, I / is a 
simple element in it, we obtain that for every nonzero l\, I2 £ R\, 

l? 2]+1 R d - [d/2] -i n lf 2]+l R d - [d /2]-i = {0} if h/h i k. (38) 
Therefore it suffices to show that for every 2-dimensional linear subspace P lying in M Rd SL2 
there is / G Ri such that P C l^ d/ ^ +1 Rd^[d/2]-i- Let /1, f 2 be a basis of P. Then (36) implies 
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that 

fi = l [ f /2]+1 hi for some Ue Ri,hi G Rd-[d/2]-i- (39) 
We have to show that if afi + (3f2 G A/" RdSL2 for every a,(3Ek, then Z1/Z2 G 

For contradiction, assume that Zi and I2 are linearly independent. Applying SL2, we then 
may assume that l\ = x, I2 = y- Since P C J\f R sl 2 ' f rom (36) we deduce that for every 
a,/3 £ k there are G k, h & Rd-[d/2]-i (depending on a, /3) such that <W d / 2 ] +1 /ii + 
f3y\- d / 2 \+ l h2 = (fix+uy)^ d / 2 ^ +1 h. Using that k[x, y] is unique factorization domain, we deduce 
from this equality and (39) that fiv 7^ if a(3 7^ 0. Hence we may assume that for every 
nonzero a, (3 G k there are fx G k, h G ^d-k/21-1 (depending on ct,/3) such that 

+ PyW%+ 1 h 2 = (fix + y^^/i. (40) 

Note that 

when a and (3 in (40) vary, ranges over an infinite set. (41) 
Indeed, otherwise (40) implies that there is a basis of P whose elements are divisible by 
some (/Lix + y)^/ 2 ^ 1 . Hence x' d//2 l +1 /ii is divisible by (px + y)[ d / 2 l +1 as well. Since deg/ii < 
[d/2] + 1, this is impossible. 

We now consider separately the cases of even and odd d. First, let d be even, d = 2m. 
Then 

hi = TT=o mx m - l - x y\ Vi G k. (42) 
Plugging (42) in equality (40), and then differentiating it m times with respect to x, sub- 
stituting y = —jix, and dividing both sides by ax m , we obtain the following equality: 

ES^-ir^f W** = 0. (43) 

Since /ii 7^ 0, (43) contradicts (41). 

Let now d be odd, <i = 2m — 1. Then /ii is still given by (42) and 

h2 = ET=o^ j V m ^\ (44) 

Plugging (42), (44) in equality (40), and then differentiating it m — 1 times respectively 
with respect to x and y, substituting y = —fix, and dividing both sides by x m , we obtain 
respectively the equalities 

e m -i» m = f ES 1 ^ 1 ^ (45) 

ar? m -i + /3 E^To 1 (-l) m " J (i^a, V™^' = 0. (46) 
Multiplying (46) by 

@m—iH m -i replacing 6 m — ifi m by the right hand side of (45), and 
dividing both sides by a, we obtain 

Vm^e^' 1 + ^-l^ grg^a ^iM™-^- 1 = 0. (47) 

From (41) we deduce that all the coefficients of the left hand side of (47), considered as a 
polynomial in vanish. In particular, 

%0m-i = %»-i0o = 0. (48) 

If 9 m -i = 0, then (45), (41) imply rjo = . . . = r/ m _i = contrary to hi 7^ 0. Similarly, 
if rj m ^i = 0, then (46) (41) imply 9q = ... = 6 m -\ = contrary to /12 7^ 0. Thus, 
rjm-idm-i / 0, whence, by (48), r] = 8 = 0. From (42), (44), (40) we then deduce that 
for m ^ 2, the left hand side of (40) is divisible by xy. Hence h in (40) is divisible by 
xy as well; in particular, m ^ 3. Thus, for m ^ 3, dividing both sides of (40) by xy, we 



12 



MARK LOSIK, PETER W. MICHOR, AND VLADIMIR L. POPOV 



obtain ax m ~ x h! x + /3y m_1 /?/ 2 = (fix + y) m ~ x h! , with h[, h' 2 , h G R m -2- This means that in 
considering (40) we may step down from case m to case m — 1. Continuing this way we 
reduce the consideration of (40) to the case m = 2. In this case, the above argument shows 
that h is a nonzero element of R\ divisible by xy. This contradiction completes the proof. 
□ 

Corollary 3.14. The action 0/SL2 on the set of maximal linear subspaces of Rd lying in 
M R , sl 2 ^ s ifo^nsitive. The dimension of every such subspace is equal to d — [d/2] and 2 
respectively for for d 7^ 1 and d = 1 . 



Theorem 3.15. For G = SL 2 and V = R dl © . . . © R, 

pol ind (V) = 



1 if di = 1 for some i, 
00 otherwise. 



Proof. Since the G-module i?o is trivial, by (33) we may assume that di ^ 1 for every i. 
Since fc[i?i] SL2 = k and fc[i?® 2 ] SL2 7^ fc, Lemma 3.11 implies that polind(i?i) = 1. From 
this, the Corollary of Lemma 3.6, and (32) we deduce the claim for the cases where di = 1 
for some i. 

Assume now that di ^ 2 for every i, and let L be a linear subspace of V lying in N VG - 
Let 7Tj : V = Rdi © ... © Rd m — ► -R^ be the natural projection to the zth summand, 
and let Lj := 7Tj(L). Since ^i{N VG ) = N R . G , we have L, C A/V G for every i. Hence 
by Lemma 3.13 and (37), for every i, there is homomorphism 7$ : G m — > G such that 

^ Rdiili)- Take now a point v € L such that 7Tj(f) 7^ for every i. Since L lies in M VG , 
it follows from (34) that v G ^(7) for some 7. Hence 7Tj(f) G ^(7). But 7Tj(v) G Rdiili) 
as well. By (37), (38), this yields Rd^i) = ^(7). Hence L C V(7). From Lemma 3.6 we 
now deduce that pol ind (V) = 00 completing the proof. □ 

Theorem 3.16. Let G be a connected semisimple algebraic group and let 2 be its Lie algebra 
endowed with the adjoint G -action. Then 



pol ind(fl) 



1 if Q is not isomorphic to © ... © 5(2, 
00 otherwise. 



Proof. In this case, M &G is the cone of all nilpotent elements in 5, see, e.g., [PV, 5.1]. Every 
subspace 9(7) is the unipotent radical of a parabolic subalgebra of g, see [B2, VIII, 4.4], 
[Sp, 8.4.5], and hence lies in a maximal (with respect to inclusion) unipotent subalgebra 
of 0. Maximal unipotent subalgebras of g are precisely the unipotent radicals of Borel 
subalgebras of g, and G acts transitively on the set of such subalgebras, see, e.g., [Sp, 
Ch. 6]. This implies that for a linear subspace L of g lying in J\f G the following properties 
are equivalent: 

(i) the subalgebra of g generated by L is unipotent (i.e., lies in M sG )\ 

(ii) there is a homomorphism 7 : G m — > G such that L C 0(7). 

From this, (32), and Lemma 3.6 we deduce that equality pol ind = 1 is equivalent to the 
following property: there is a 2-dimensional linear subspace L of such that L lies in J\f g G 
but the subalgebra of generated by L does not lie in Af G . If this property holds, we say, 
for brevity, that is a 2-algebra. 

We shall show now that if is not isomorphic to SI2 © . . . ©fife, then is a 2-algebra. To 
this end we remark that if a semisimple subalgebra s of is a 2-algebra, then is a 2-algebra 
as well: since the cone of nilpotent elements of s lies in Af gG , this readily follows from the 
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definition of a 2-algebra. Given this remark, we see that the foliowing two statements 
immediately imply our claim: 

(a) if g % © • • • © sb, then g contains a subalgebra isomorphic to either s[3 or 505; 

(b) 5(3 and SO5 are 2-algebras 

(note that in SO5 there are no subalgebras isomorphic to SI3, and vice versa). 

To prove (a), denote by <3? the root system of g with respect to a fixed maximal torus. 
Let ai,... ,ai be a system of simple roots in <3? (enumerated as in [Bi]). Fix a Chevalley 
basis {X a , X_ a , H a } a& ^> of g, [B2]. We may assume that g is simple, g ^ 5I2, SO5. For such 
g, it is easily seen that there are two roots A, \i G <!> such that the subalgebra of g generated 
by X\ and X^ is isomorphic to SI3: for g of types A; (I ^ 2), (/ ^ 3), Q (/ ^ 3), 
(I ^ 4), F4, one can take A = a±, fi = 02; for types Eq, E7, Eg, take A = ot\, fi = 03; for 
type G2, take A = Q2, // = 3«i + ol<i . This proves (a). 

We turn now to the proof of (b). In SI3 we explicitly present a subspace L enjoying the 
desired properties (we are grateful to H. Radjavi for this example, [R]). Namely, take 



L :— {X ai +X a , 2 , X- ai — X- a2 ) — I 



a 
6 0a 
0-6 



a, 6 G 



(49) 



a 2 • 



Then (49) implies that the subalgebra generated by L contains the element H ai — H, 
Since it is semisimple, this subalgebra does not lie in 7V g G . On the other hand, the matrix 
in the right hand side of (49) is nilpotent (this is equivalent to the property that the sums 
of all its principal minors of orders 2 and 3 are equal to 0, and this is immediately verified). 
So, L C M gG . This proves (b) for s[ 3 . 

Let now g = SO5. In this case, an explicit construction of the desired subspace L is 
unknown to us, so we shall use an indirect argument. The underlying space of g is the 
space of all skew-symmetric 5 x 5-matrices. Let Xij € k[g], 1 ^ i,j ^ 5, be the standard 
coordinate functions on g given by Xy((a pg )) = <%■■ Then Xij = —Xji. Consider the matrix 
A := (xij). Then k[g] G = A^/2,/4] where /2, fi are the coefficients of the characteristic 
polynomial of A, i.e., det(i/5 — A) = t 5 + f2t 3 + f^t, see, e.g., [PV]. The Newton formulas 
expressing the sums of squares of eigenvalues of A via the elementary symmetric functions 
of them imply that tr(^ 2 ) = -2/ 2 , tr(,4 4 ) = 2/| - 4/ 4 . Hence 

k[gf = k[ti(A 2 )MA% (50) 

Let now yij, Zij G /c[g® 2 ], 1 ^ i,j ^ 5, be the standard coordinate functions on g® 2 given 
by Uij((a P q), (Ws)) = (Hj, Zij((a pq ), (6 rs )) = 6^-. Then y^ = -y^ and = -zji. Consider 
the matrices B := (yij), C := (%). Taking into account that tr(PQ) = tr(QP) for any 
square matrices P, Q, it is not difficult to deduce that for every 01,02 G k, the following 
equalities hold: 

tr((ai5 + a 2 C) 2 ) = a 2 tr(5 2 ) + 2aiQ 2 tr(5C) + a 2 tr(C 2 ), 



(51) 
(52) 



tr((aiB + a 2 C) 4 ) = aftr{B 4 ) + 4a?a 2 tr( J B 3 C) 

+ 2a 2 a 2 ,(2tr( J B 2 C 2 ) + tr({BC) 2 )) + 4 ai alti{BC 3 ) + a 4 2 tr(C 4 ). 

From (50), the definition of pol 2 /c[g] G (see Example 2.5 and the first paragraph right 
after it), and (51), (52) we deduce that pol 2 /c[g] G is the algebra 

k[tr(B 2 ), tv(BC), tr(C 2 ), tr(B 4 ), tv(B 3 C), 2tv(B 2 C 2 ) + tx((BC) 2 ), ti(BC 3 ), tr(C 4 )]. 
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This shows that the transcendence degree of pol 2 /c[g] G over k is not bigger than 8. On the 
other hand, since dimg® 2 = dimG = 20, the transcendence degree of fc[g® 2 ] G over k is 
not smaller than dimg® 2 — dimG = 10, see, e.g., [PV, Theorem 3.3 and the Corollary of 
Lemma 2.4] (actually it is equal to 10 since, as one easily proves, the generic G-stabilizer 
of the G-module g® 2 is finite). Therefore fc[g® 2 ] G is not integral over pol 2 fc[£|] G . By (32) 
and the Corollary 3.5 we now deduce that pol ind (g) = 1, i.e., g is a 2-algebra. 

To complete the proof we have to calculate pol ind (g) for g = gi © . . . ffig m where Qi = 5I2 
for every i. We may assume that G = G\ x . . . x G m where Gi = SL 2 for every i. The 
Corollary of Lemma 3.6 then reduces the proof to the case m = 1. Since the SL2-modules 
5(2 and i?2 are isomorphic, the claim now follows from Theorem 3.15. □ 

Call a linear subspace L of a reductive Lie algebra g triangularizable if there is a Borel 
subalgebra b of g such that L lies in the unipotent radical of b (for g = Mat nX n, this means 
that L is conjugate to a subspace of the space of upper triangular matrices, see [G], [CRT], 
[MOR]). Call L nilpotent if every element of L is nilpotent. 

Corollary 3.17. A semisimple Lie algebra g contains a 2- dimensional nilpotent nontrian- 
gularizable linear subspace if and only if g is not isomorphic to © . . . ©sb- 
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